System and method for measuring reading skills

ABSTRACT

A system and method for measuring reading skills is described. An individual whose reading skills are to be evaluated reads aloud from a text. As the person reads aloud from the text, a speech signal is captured. The speech signal is analyzed to provide an estimate of what the individual said and to measure a timing of the words said. The estimate and timing is combined with parameters assigned to each word said to form a measure of the individual&#39;s reading skill. The measure of the individual&#39;s reading skill is substantially independent of the text.

RELATED APPLICATIONS

The present patent application claims priority under 35 U.S.C. § 119(e)to U.S. Provisional Patent Application Ser. No. 60/585,656, which wasfiled Jul. 6, 2004. The full disclosure of U.S. Provisional PatentApplication Ser. No. 60/585,656 is incorporated herein by reference.

FIELD

The present invention relates generally to measuring reading skills, andmore particularly, relates to using a standardized scale to provide ameasure of an individual's reading skills that is independent ofmaterial being read by the individual.

BACKGROUND

Interactive language proficiency testing systems using speechrecognition are known. For example, U.S. Pat. No. 5,870,709, issued toOrdinate Corporation, describes such a system. In U.S. Pat. No.5,870,709, the contents of which are incorporated herein by reference,an interactive computer-based system is shown in which spoken responsesare elicited from a subject by prompting the subject. The prompts maybe, for example, requests for information, a request to read or repeat aword, phrase, sentence, or larger linguistic unit, a request tocomplete, fill-in, or identify missing elements in graphic or verbalaggregates, or any similar presentation that conventionally serves as aprompt to speak. The system then extracts linguistic content, speakerstate, speaker identity, vocal reaction time, rate of speech, fluency,pronunciation skill, native language, and other linguistic, indexical,or paralinguistic information from the incoming speech signal.

The subject's spoken responses may be received at the interactivecomputer-based system via telephone or other telecommunication or datainformation network, or directly through a transducer peripheral to thecomputer system. It is then desirable to evaluate the subject's spokenresponses and draw inferences about the subject's abilities or states.

Although interactive language proficiency testing systems provide manyimportant features, there continues to be room for new features andimprovements. One area in which there is room for improvement relates tocreating a standardized scale for measuring reading skills that isindependent of the material read by the subject. By measuring readingskills in a manner such that the material being read does not impact thescore, a more reliable reading skills measure may be obtained.Accordingly, it would be beneficial to have a way to measure readingskills that is independent of the material read by the subject.

SUMMARY

A system and method for measuring reading skills is described. A userreads aloud from a source text. The source text includes units of text,such as letter strings, pseudo-words, words, phrases, sentences,paragraphs, and extended passages. For example, a letter string may forma sub-word string, such as <ght> in “caught” or “lighten” or apseudo-word, such as “strale” or “kaffish.” Each unit has a set ofparameters that characterize the unit. The set of parameters for eachunit includes salient linguistic and orthographic features of thepresentation context and item response difficulties for this context.Additionally, the set of parameters for each unit includes a durationmodel specific to the unit of text.

A speech signal is formed when the user is reading aloud. The speechsignal is captured either directly or via a recording of the speechsignal. The speech signal is analyzed. An estimate of what theindividual said when reading aloud is calculated. Additionally, latencyand accuracy for each unit of text read is extracted. The accuracyincludes the accuracy in word recognition, decoding, and oral reading. Arate based on time for reading each unit of text read from the sourcetext is also measured. By combining the estimate of the phonologicalform, the extracted latency and accuracy, the time, and the set ofparameters for each unit of text read, a measure of the individual'sreading skill can be calculated. This measure of the individual'sreading skill is substantially independent of the source text.

In one example, the measure is based on word-level statistics treatingeach word as a single “item.” For each word the following information isextracted: whether the word was correctly read; the time taken to decodeand read the word; the presence of false starts, hesitations, or otherfiller; overall articulation rate of the speaker; and inherent“difficulty” of the item.

It is also possible to combine the measure of the individual's readingskill with comprehension evidence, such as through the use of secondaryquestions, to ascertain whether the user comprehended the source text.

These as well as other aspects and advantages will become apparent tothose of ordinary skill in the art by reading the following detaileddescription, with reference where appropriate to the accompanyingdrawings. Further, it is understood that this summary is merely anexample and is not intended to limit the scope of the invention asclaimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Presently preferred embodiments are described below in conjunction withthe appended drawing figures, wherein like reference numerals refer tolike elements in the various figures, and wherein:

FIG. 1 is a block diagram of a system for measuring reading skills,according to an example;

FIG. 2 is a flow diagram of a method for measuring reading skills,according to an example; and

FIG. 3 is a flow diagram of a method for measuring reading skills,according to another example.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a system 100 for measuring reading skills.The system 100 interacts with a user 102 whose reading skills are to bemeasured and includes a computing platform 104. While FIG. 1 depicts adirect connection between the user 102 and the computing platform 104,there may be a network and/or other entities connecting the user 102 andthe computing platform 104.

The user 102 may be, for example, a student (child or adult) in a formaleducation program, a job applicant seeking employment requiring acertain level of reading proficiency, or someone who is interested inknowing his or her reading skill level for any reason. For example, theuser 102 may be learning how to read and measuring improvement inreading skill may provide useful information regarding the user'sprogress.

The user 102 reads aloud from a source text. The source text may be anycombination of units. The units may be letter strings, pseudo-words,words, phrases, sentences, paragraphs, extended passages, and so on.Preferably, the units are words. The user 102 may read aloud from thesource text in a manner such that the computing platform 104 can detectspeech signals as the user 102 reads aloud. Alternatively, the speechsignals may be recorded as the user 102 reads aloud, and a recording ofthe user's responses may be presented to the computing platform 104. Thecomputing platform 104 may capture the speech signals.

The computing platform 104 may be any combination of hardware, software,and/or firmware. The computing platform 104 is shown as a simplerectangular box in FIG. 1 to emphasize the variety of different formsthe computing platform 104 may take on from one example to the next. Inthe illustrated form, the computing platform 104 includes a speechrecognition system 106, an evaluation device 108, and a calculationdevice 110. While the speech recognition system 106, the evaluationdevice 108, and the calculation device 110 are shown as separateentities in FIG. 1, two or more of the speech recognition system 106,the evaluation device 108, and the calculation device 110 may becombined into a single entity.

The computing platform 104 may include additional entities as well, suchas an input device, an output device, and memory. Input devices mayinclude a mouse, a keyboard, and a microphone. Output devices mayinclude a display, a speaker, and a printer. The memory may includevolatile and/or non-volatile memory devices. Additionally, the memorymay be located on a memory chip on a printed circuit board or located ona magnetic or optical drive disk.

The speech recognition system 106 may be any combination of hardware,software, and/or firmware. Preferably, the speech recognition system 106is implemented in software. For example, the speech recognition system106 may be the HTK software product, which is owned by Microsoft and iscurrently available for free download from the Cambridge UniversityEngineering Department's web page (http://htk.eng.cam.ac.uk). The speechrecognition system 106 may receive signals representing the speech ofthe user 102 who is reading the source text aloud.

The speech recognition system 106 may be an automatic speech recognitionsystem that operates by recognizing and aligning responses to provide anestimate of the speech. The calculated estimate may be an estimate oflinguistic content of the speech and may be in the form of a data streamthat represents the user's speech. The linguistic content of speech mayinclude a distinctive feature, a segment, a phoneme, a syllable, amorpheme, a word, a syntactic phrase, a phonological phrase, a sentence,a paragraph, and an extended passage. For example, the output of thespeech recognition system 106 may be a sequence of words in a machinerecognizable format, such as American Standard Code for InformationInterchange (ASCII).

The evaluation device 108 may be any combination of hardware, software,and/or firmware. Preferably, the evaluation device 108 is implemented insoftware. The evaluation device 108 may extract latency and accuracy foreach unit of text read by the user. The evaluation device 108 measures atime for each unit of text read. The time may be measured between an endof one unit of text read and an end of another unit of text read. Themeasured time may be scaled to account for variations in the user'sarticulation rate. The scaling of the measured time may be performedusing a duration model, which is a model of expected duration of alinguistic form of a unit of text. The linguistic form may includephonological structure, morphological structure, lexical structure,stochastic structure, and/or syntactic structure of the text units.

The duration model may be generated by analyzing a sample ofrepresentative users that are known “good” readers and measuringstatistics of durations. The measured statistics may be used to create amodel (i.e., the duration model) that predicts how deviant a givenduration is. A deviant duration is typically longer than the modeldurations. The duration model, a text model, and each individualobservation may be used to create an estimate of the reader's ability.

The calculation device 110 may be any combination of hardware, software,and/or firmware. Preferably, the calculation device 110 is implementedin software. The calculation device 110 combines the estimate of whatthe user said when reading the source text, the measurement of time foreach unit of text read, and a set of parameters assigned to each unit oftext read. This combination may be used to form a measure of the user'sreading skill.

The set of parameters for each unit of text in the source text may beincluded in the calculation device 110. Alternatively, the set ofparameters may be provided to the calculation device 110 by anotherdevice located within the computing platform 104 or remotely. The set ofparameters for each unit of text may be calculated using statisticalanalysis, such as Item Response Theory, to evaluate the units of text.Details on Item Response Theory may be found in “Introduction toClassical and Modern Test Theory,” authored by Linda Crocker and JamesAlgina, Harcourt Brace Jovanovich College Publishers (1986), Chapter 15;and “Best Test Design; Rasch Measurement,” by Benjamin D. Wright andMark H. Stone, Mesa Press, Chicago, Ill. (1979), the contents of both ofwhich are incorporated herein by reference.

The set of parameters for each unit includes salient linguistic andorthographic features of the presentation context and item responsedifficulties for this context. Additionally, the set of parameters mayinclude a duration model for each unit of text. The set of parametersmay be based on an analysis of speech formed by a plurality ofindividuals reading each unit of text in a similar context. The similarcontext relates to the linguistic structure of any superordinatelinguistic unit and/or to the probability of the unit occurring within aword sequence that includes the unit. Alternatively, the set ofparameters may be based on an analysis of speech formed by a pluralityof individuals reading each unit of text in various contexts. In thisexample, the analysis may include identifying similarities within thespeech.

The plurality of individuals may have a known set of characteristics,such as demographic characteristics and skill-level characteristics. Thedemographic characteristics may include age, gender, race, ethnicity, aswell as other characteristics. The skill-level characteristics mayinclude spoken language proficiency, reading comprehension skill,educational achievement, vocabulary skill, as well as othercharacteristics.

The set of parameters for each unit may also include any superordinatelinguistic unit within which the unit occurs. Thus, for example, aparameter of a word can be a structural schema relating to the nounphrase within which the unit occurs. This example may enable theparametric model to more accurately estimate reading skill for a worditem by using schematic context to adjust the expected elapsed time forthe word.

Where any or all of the speech recognition system 106, the evaluationdevice 108, and the calculation device 110 are implemented in software,the computing platform 104 will typically be associated with a generalpurpose or application specific processor and memory. In addition, thecomputing platform 104 may be coupled to or include one or more inputand/or output devices, such as a keyboard, microphone, speaker, display,etc. For a computing platform 104 that includes or is coupled to adisplay, the display may present the source text to the user 102.Alternatively, the user 102 may read aloud from a source text that isindependent from the computing platform 104, such as a book or pamphlet,although in such cases the source text needs to be identified to thecomputing platform 104.

FIG. 2 is a flow diagram of a method 200 for measuring reading skills.At block 202, a speech signal is captured. The speech signal may becaptured when the speech signal is formed as the user 102 reads aloudfrom a source text. The speech signal may be captured directly by thespeech recognition system 106 or may be recorded first and then providedto the speech recognition system 106.

The source text may be formed by units. The units may be a subset of thetext, such as letter strings, words, phrases, sentences, paragraphs, andextended passages. The source text may be designed to have a difficultylevel. The difficulty level of the source text may remain the same orvary throughout the text. For example, the difficulty level of thesource text may increase as the user 102 reads aloud from the sourcetext.

At block 204, an estimate of speech is calculated. The speechrecognition system 106 may calculate an estimate of the speech. Thecalculated estimate may be an estimate of the linguistic content of thespeech and may be in the form of a data stream that represents theuser's speech. For example, the output of the speech recognition system106 may be a sequence of words in a machine recognizable format, such asASCII.

At block 206, a time for each unit of text read is measured. Theevaluation device 108 may measure the elapsed time for each unit of textread. The time may be measured between an end of one unit of text readand an end of another unit of text read. The measured time may be scaledto account for variations in the user's articulation rate.

At block 208, a measure of the individual's reading skill is formed. Themeasure of the individual's reading skill may be substantiallyindependent of the source text. The measure of the user's reading skillmay be formed by combining the estimate of the speech, the measurementof time for each unit of text read, and a set of parameters for eachunit of text read. The set of parameters for each unit includes salientlinguistic and orthographic features of the presentation context anditem response difficulties for this context. Additionally, the set ofparameters may include a duration model for each unit of text and anysuperordinate linguistic unit within which the unit occurs.

FIG. 3 is a flow diagram of a method 300 for measuring reading skillsaccording to another example. At block 302, text is presented to anindividual. The individual is the user 102 whose reading skill is to bemeasured. The text may be presented to the individual in a writtenformat, such as text on a piece of paper, or in an electronic format,such as on a computer monitor. The text may be comprised of words, andhave a constant or varying difficulty level.

At block 304, the individual reads the text and the individual'sresponses are recorded. The individual's responses may be recorded byany recording device, such as a tape recorder. The recording device maybe integrated into the computing platform 104 or may be a stand-alonedevice. If the recording device is a stand-alone device, the responsesmay be presented to the computing platform 104, which may be detected bythe speech recognition system 106.

At block 306, the responses are analyzed. The response may be analyzedby an automatic speech recognition system, such as the speechrecognition system 106. The responses may be analyzed based on a set ofparameters defined for each word in the text, timing of the response,accuracy of the response, and characteristics of the individual.

The set of parameters for each unit includes salient linguistic andorthographic features of the presentation context and item responsedifficulties for this context. Additionally, the set of parameters mayinclude a duration model for each unit of text. The timing of theresponse may be calculated by measuring the time between the end of oneword and the end of another word read. The accuracy of the response maybe determined by the speech recognition system 106.

The characteristics of the individual may include demographiccharacteristics and skill-level characteristics. The demographiccharacteristics may include age, gender, race, ethnicity, as well asother characteristics. The skill-level characteristics may includespoken language proficiency, reading comprehension skill, educationalachievement, vocabulary skill, as well as other characteristics.

At block 308, a measure of the individual's reading skill is calculated.The measure of the individual's reading skill may be substantiallyindependent of the text. The measure of the user's reading skill may bebased on the analysis of the set of parameters defined for each word inthe text, the timing of the response, the accuracy of the response, andthe characteristics of the individual.

By measuring reading skills in the described manner, an estimate of anindividual's reading skill can be estimated such that the estimate isindependent of the source text. So if an individual's reading skill isevaluated multiple times in a short time frame using different sourcetexts, the individual may receive a substantially similar reading skillmeasurement for each of the source texts read. Accordingly, a readingskill scale can be formed that is substantially independent of thematerial read.

Further, the reading skill measurement may be calculated by analyzing anindividual's response when reading aloud for a short period of time. Asa result, a reliable reading skill measurement may be obtained withminimal inconvenience to the individual.

It is also possible to combine the measure of the individual's readingskill with comprehension evidence to ascertain whether the usercomprehended the source text. Secondary questions may be used todetermine a user's level of comprehension. For example, the individualmay be asked a series of questions regarding the content of the sourcetext. Based on the user's responses to the questions, a user'scomprehension may be ascertained.

It should be understood that the illustrated embodiments are examplesonly and should not be taken as limiting the scope of the presentinvention. The claims should not be read as limited to the describedorder or elements unless stated to that effect. Therefore, allembodiments that come within the scope and spirit of the followingclaims and equivalents thereto are claimed as the invention.

1. A method for measuring reading skills of a plurality of individuals on a single scale, comprising in combination: capturing a speech signal formed when an individual reads aloud from a source text; estimating linguistic content of what the individual said when reading aloud; extracting latency and accuracy for units of text read from the source text; measuring elapsed time for the units of text read from the source text; combining the estimated linguistic content, the extracted latency and accuracy, the elapsed time, and a set of parameters for the units of text read from the source text to form a measure of the individual's reading skill that is substantially independent of the source text.
 2. The method of claim 1, wherein the linguistic content is selected from the group consisting of a distinctive feature, a segment, a phoneme, a syllable, a morpheme, a word, a syntactic phrase, a phonological phrase, a sentence, a paragraph, and an extended passage.
 3. The method of claim 1, wherein a unit of text is selected from the group consisting of a letter string, a word, a phrase, a sentence, a paragraph, and an extended passage.
 4. The method of claim 1, wherein the elapsed time is measured between an end of one unit of text read and an end of another unit of text read.
 5. The method of claim 1, wherein the elapsed time for the units of text read is scaled to account for variations in the individual's articulation rate.
 6. The method of claim 1, wherein the elapsed time for the units of text read is scaled according to a duration model that depends on a linguistic form of the units of text read, wherein the linguistic form of the units of text read includes structure selected from the group consisting of phonological, morphological, lexical, stochastic, and syntactic.
 7. The method of claim 1, wherein the set of parameters for the units of text read includes at least one of an item response theory difficulty, a duration model for the units of text read, and any superordinate linguistic unit in which the units of text read occur.
 8. The method of claim 1, wherein the set of parameters for the units of text read is based on analysis of speech produced by a plurality of individuals having known characteristics selected from the group consisting of demographic characteristics and skill-level characteristics.
 9. The method of claim 8, wherein the plurality of individuals read the units of text in a similar context including at least one of a linguistic structure of any superordinate linguistic unit and probability of the text occurring within a word sequence that includes the text.
 10. A method for measuring reading skills, comprising in combination: presenting text to a individual whose reading skill is to be measured; recording responses as the individual reads the text aloud; analyzing the responses based on a set of parameters defined for words in the text, timing of the response, and accuracy of the response; and calculating a measure of the individual's reading skill based on the analysis, wherein the measure of the individual's reading skill is substantially independent of the text.
 11. The method of claim 10, wherein analyzing the responses includes an automatic speech recognition system performing the analysis.
 12. The method of claim 10, wherein the set of parameters defined for the words in the text is based on analysis of speech formed by a plurality of individuals reading the words in the text in a similar context, wherein the similar context includes at least one of a linguistic structure of any superordinate linguistic unit and probability of the text occurring within a word sequence that includes the text.
 13. The method of claim 10, wherein the set of parameters defined for the words in the text include at least one of an item response theory difficulty, a duration model, and any superordinate linguistic unit within which the word occurs.
 14. The method of claim 10, further including analyzing the responses based on characteristics of the individual, wherein the characteristics of the individual are selected from the group of characteristics consisting of demographic characteristics and skill-level characteristics.
 15. A system for measuring reading skills, comprising in combination: a processor; data storage; and machine language instructions stored in the data storage executable by the processor to: capture a speech signal formed when an individual reads aloud from a source text; estimate linguistic content of what the individual said when reading aloud; extract latency and accuracy for units of text read from the source text; measure elapsed time for the units of text read from the source text; and combine the estimate of linguistic content, the extracted latency and accuracy, the elapsed time, and a set of parameters for the units of text read from the source text to form a measure of the individual's reading skill that is substantially independent of the source text.
 16. The system of claim 15, wherein the linguistic content is selected from the group consisting of a distinctive feature, a segment, a phoneme, a syllable, a morpheme, a word, a syntactic phrase, a phonological phrase, a sentence, a paragraph, and an extended passage.
 17. The system of claim 15, wherein the units of text are selected from the group consisting of a letter string, a word, a phrase, a sentence, a paragraph, and an extended passage.
 18. The system of claim 15, wherein the elapsed time is measured between an end of one unit of text read and an end of another unit of text read.
 19. The system of claim 15, wherein the elapsed time for the units of text read is scaled to account for variations in the individual's articulation rate.
 20. The system of claim 15, wherein the elapsed time for the units of text read is scaled according to a duration model that depends on a linguistic form of the units of text read, wherein the linguistic form of the units of text read includes structure selected from the group consisting of phonological, morphological, lexical, stochastic, and syntactic.
 21. The system of claim 15, wherein the set of parameters includes at least one of an item response theory difficulty, a duration model for the units of text read, and any superordinate linguistic unit in which the units of text read occurs.
 22. The system of claim 15, wherein the set of parameters for the units of text read is based on analysis of speech produced by a plurality of individuals reading the units of text in a similar context, wherein each of the plurality of individuals reading the source text has known characteristics selected from the group consisting of demographic characteristics and skill-level characteristics.
 23. The system of claim 15, wherein the similar context includes at least one of a linguistic structure of any superordinate linguistic unit and probability of the text occurring within a word sequence that includes the text. 